Differing region detection system and differing region detection method

ABSTRACT

The present invention enables detection of a local differing region between images. Inter-image difference information indicating a difference in feature amounts for each subregion between first and second images is generated based on a first feature amount vector that is a set of feature amounts respectively corresponding to a plurality of subregions in the first image and a second feature amount vector that is a set of feature amounts respectively corresponding to a plurality of subregions in the second image, a differing region that is an image region that differs between the first and second images is detected based on differences in the respective subregions indicated by the inter-image difference information, and detection information that indicates a result of the detection is outputted.

BACKGROUND

The present invention relates to a differing region detection system anda differing region detection method.

The recent proliferation of video sites on the Internet has resulted ina deluge of illegal videos created from original videos, which hasbecome a social issue. Examples of illegal videos include videos thatare exact copies of an original video, videos created by extracting apart of an original video such as a highlight scene, and videos createdby modifying an original video by adding a telop or the like thereto. Inconsideration thereof, there are demands for detection of a video havingidentity with a source video while also taking such modified videos intoconsideration.

For example, Patent Document 1 discloses a method of judging identitybetween two images. Specifically, with the method disclosed in PatentDocument 1, a multidimensional feature amount vector is generated foreach image, and feature amount vectors are compared between images inorder to judge identity between the images. By applying such a methodto, for example, a part of frame images in a video, identity betweenvideos can be judged even if the videos has been modified.

Patent Document 1: WO 2010/084714

However, although the method disclosed in Patent Document 1 enablesidentity between videos to be judged as described above, when there is alocal difference such as presence/absence of telops or a difference incontents of telops between videos judged to have identity, the methoddisclosed in Patent Document 1 does not enable a determination to bemade regarding where the differing region is. For example, even if atelop-added video is judged by the method described above to be a videohaving identity with a source video, the video judged to have identitymust be played back in order to check where the telop has been added,resulting in an significant increase in work load.

SUMMARY

The present invention has been made in consideration of suchcircumstances and an object thereof is to detect a local differingregion between images.

A differing region detection system according to an aspect of thepresent invention comprises: a difference information generating unitconfigured to generate inter-image difference information indicating adifference in feature amounts for each subregion between first andsecond images based on a first feature amount vector that is a set offeature amounts respectively corresponding to a plurality of subregionsin the first image and a second feature amount vector that is a set offeature amounts respectively corresponding to a plurality of subregionsin the second image; and a differing region detecting unit configured todetect a differing region that is an image region that differs betweenthe first and second images, based on differences in the respectivesubregions indicated by the inter-image difference information, andoutput detection information indicating a result of the detection.

Moreover, as used in the present invention, the term “unit” not onlysignifies physical means but also includes cases where functions of the“unit” are realized by software. In addition, functions of one “unit” ordevice may be realized by two or more physical means or devices, andfunctions of two or more “units” or devices may be realized by onephysical means or device.

According to the present invention, a local differing region betweenimages can be detected.

DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing a configuration of a differing regiondetection system that is an embodiment of the present invention;

FIG. 2 is a diagram showing an example of a differing region detected bythe differing region detection system;

FIG. 3 is a diagram showing an example of region splitting of a frameimage;

FIG. 4 is a diagram showing an image from which M-th dimension featureamounts are extracted;

FIG. 5 is a diagram showing an example of a feature amount vector storedin a feature amount storing unit;

FIG. 6 is a diagram showing an example of a feature amount vector storedin a feature amount DB;

FIG. 7 is a diagram showing a configuration example of a differenceinformation generating unit;

FIG. 8 is a diagram showing an example of generation of a differencevector by the difference information generating unit;

FIG. 9 is a diagram showing an example of a difference vector stored ina difference information generating unit;

FIG. 10 is a diagram showing a configuration example of a differingregion detecting unit;

FIG. 11 is a diagram showing an example of difference value mapping anddiffering region detection;

FIG. 12 is a diagram showing an example of detection information storedin a detection information storing unit;

FIG. 13 shows an example of a screen which identifiably displays asegment in which a local modification has been detected;

FIG. 14 shows an example of a screen which identifiably displays aposition of a differing region;

FIG. 15 shows an example of a screen which identifiablythree-dimensionally displays a position of a differing region;

FIG. 16 shows an example of an output screen of a detection result of adiffering region in a case where there is a plurality of original videosas source candidates;

FIG. 17 shows an example of an output screen of a detection result of adiffering region in a case where there is a plurality of original videosas source candidates;

FIG. 18 shows an example of an output screen in a case where shopsplitting is performed in consideration of a local modification; and

FIG. 19 is a flow chart showing an example of a differing regiondetecting process.

DETAILED DESCRIPTION

Hereinafter, an embodiment of the present invention will be describedwith reference to the drawings.

FIG. 1 is a diagram showing a configuration of a differing regiondetection system that is an embodiment of the present invention. Adiffering region detection system 10 is a system that detects a regionwith a local difference between videos having identity, and isconfigured so as to comprise a feature amount extracting unit 20, afeature amount storing unit 21, a difference information generating unit22, a difference information storing unit 23, a differing regiondetecting unit 24, a detection information storing unit 25, and adetection result output unit 26. In the present embodiment, a localdifference may also be referred to as a “local modification”. Inaddition, the differing region detection system 10 refers to a featureamount database (DB) 30 and a video database (DB) 32. Moreover, thediffering region detection system 10 is configured using one or aplurality of information processing devices, and the feature amountextracting unit 20, the difference information generating unit 22, thediffering region detecting unit 24, and the detection result output unit26 can be realized by having a processor execute a program stored in amemory. In addition, the feature amount storing unit 21 and thedifference information storing unit 23 can be realized using a storagearea of a memory, a storage device, or the like.

FIG. 2 is a diagram showing an example of a differing region detected bythe differing region detection system 10. In this case, a differingregion refers to a region in which a local modification has occurred.FIG. 2 shows videos 40 and 42 which have identity. The video 40 is, forexample, an advertisement video of an automobile that is about to belaunched, and a subtitle reading “On Sale March 1!” is displayed in alower image region 44 of a frame image constituting the video 40. On theother hand, the video 42 is an advertisement video of the sameautomobile to be broadcasted after the automobile goes on sale. As such,a subtitle displayed in a lower image region 46 of a frame imageconstituting the video 42 reads “Now On Sale!” Since the two videos 40and 42 only differ from each other in contents of the subtitles, thevideos overall are judged to have identity. In addition, with thediffering region detection system 10, a differing region 48 between thevideos 40 and 42 can be detected. In a similar manner, for example,regarding an illegal video generated from an original video, thediffering region detection system 10 is capable of detecting a differingregion that is a region in which a modification has been made to anoriginal video.

Returning to FIG. 1, various units which constitute the differing regiondetection system 10 will now be described in detail.

The feature amount extracting unit 20 extracts a feature amount vectorfrom each of a plurality of frame images that constitutes an input videoand stores the feature amount vectors in the feature amount storing unit21. In this case, for example, an input video refers to a video on theair or a video uploaded to a video site. A feature amount vector is aset of N-number (where N≧2) of feature amounts corresponding to N-numberof subregions defined in a frame image and can be generated accordingto, for example, a method described in WO 2010/084714. Each subregioncorresponding to each dimension of a feature amount vector includes, forexample, a plurality of subregions in a frame image. In addition, afeature amount of each dimension can be generated based on, for example,a difference among feature amounts of a plurality of subregionscorresponding to each dimension.

FIG. 3 is a diagram showing an example of region splitting of a frameimage. As shown in FIG. 3, for example, each frame image can be splitinto 32×32=1024 regions (split regions). A subregion corresponding toeach dimension in a feature amount vector is constituted by acombination of one or more split regions.

FIG. 4 is a diagram showing an image from which M-th dimension featureamounts are extracted. In the example shown in FIG. 4, two subregions 62and 64 correspond to the M-th dimension. In addition, the feature amountextracting unit 20 is capable of generating an M-th dimension featureamount based on a difference between a feature amount (region featureamount) of the subregion 62 and a feature amount (region feature amount)of the subregion 64. In this case, the feature amounts of the respectivesubregions 62 and 64 can be calculated by any method such as a methodbased on an average value or a median value of pixel values in therespective subregions. In addition, the feature amount extracting unit20 can generate an M-th dimension feature amount by quantizing adifference between the region feature amounts of the subregions 62 and64 into three values (−1, 0, 1). By generating a feature amount for eachof the dimensions (first to N-th dimensions), the feature amountextracting unit 20 can generate an N-th dimension feature amount vector.It should be noted that the method of calculating a feature amount ofeach dimension as described above is merely an example and any methodmay be used as long as a feature amount vector is generated based on afeature amount of a subregion set for each dimension.

FIG. 5 is a diagram showing an example of a feature amount vector storedin the feature amount storing unit 21. As shown in FIG. 5, a featureamount vector is stored in association with a video identifier whichidentifies an input video and sequence information which indicates achronological order of a frame image. In this case, a video identifieris for identifying a batch of videos and, for example, a video title, aprogram name, a file name, or a URL (uniform resource locator) can beused. In addition, sequence information may be any information whichenables an order of a feature amount vector to be assessed and, forexample, a frame number can be used. Moreover, a video identifier neednot be used if there is only one inputted video. On the other hand, if astorage structure of data or the like enables a chronological sequenceof a feature amount vector to be identified, sequence information neednot be used.

Returning to FIG. 1, the difference information generating unit 22compares a feature amount vector of an input video stored in the featureamount storing unit 21 with a feature amount vector stored in thefeature amount DB 30 and generates a difference vector from featureamount vectors of videos with identity. Moreover, the differenceinformation generating unit 22 is also capable of generating adifference vector by comparing feature amount vectors stored in thefeature amount DB 30. In other words, with the differing regiondetection system 10, a differing region can also be detected among aplurality of videos whose feature amount vectors are stored in thefeature amount DB 30.

FIG. 6 is a diagram showing an example of a feature amount vector storedin the feature amount DB 30. The feature amount DB 30 stores featureamount vectors of a plurality of videos to be comparison objects with aninput video. In the present embodiment, a video whose feature amountvector is stored in the feature amount DB 30 will be referred to as anoriginal video. As shown in FIG. 6, a feature amount vector of anoriginal video is stored in associated with a video identifier thatidentifies the original video, a creation date/time of the originalvideo, and sequence information which indicates a chronological order ofa frame image.

FIG. 7 is a diagram showing a configuration example of the differenceinformation generating unit 22. As shown in FIG. 7, the differenceinformation generating unit 22 can be configured so as to include afeature amount comparing unit 70, a frame selecting unit 72, and adifference information output unit 74.

For example, the feature amount comparing unit 70 compares a featureamount vector of an input video with a feature amount vector in thefeature amount DB 30 for each frame. The frame selecting unit 72 selectsa frame image judged to have identity between the input video and theoriginal video based on a result of a comparison by the feature amountcomparing unit 70. Moreover, a judgment of identity between frame imagescan be performed by, for example, comparing the number of dimensionswith identical feature amounts or the number of dimensions withnon-identical feature amounts between two feature amount vectors orcomparing sizes of the two feature amount vectors. The differenceinformation output unit 74 outputs difference region informationindicating a dimension in which a difference of a feature amount isgreater than a predetermined criterion. Specifically, the differenceinformation output unit 74 generates a difference vector from a featureamount vector of a frame image selected by the frame selecting unit 72and stores the difference vector in the difference information storingunit 23.

FIG. 8 is a diagram showing an example of generation of a differencevector by the difference information generating unit 22. In thedifference vector example shown in FIG. 8, between the feature amountvector of the input video and the feature amount vector of the originalvideo, dimensions with the same feature amount are denoted by “0” anddimensions with different feature amounts are denoted by “1”. In otherwords, a difference vector is a set of difference region information ofthe respective dimensions. Moreover, the difference vector shown in FIG.7 is merely an example and different values in accordance with amagnitude of difference in feature amounts may be set to the respectivedimensions of a difference vector.

FIG. 9 is a diagram showing an example of a difference vector stored inthe difference information storing unit 23. As shown in FIG. 9, adifference vector is stored in association with video identifiers of aninput video and an original video as well as sequence information. Asshown in FIG. 9, video identifiers or sequence information may differbetween an input video and an original video. In addition, sequenceinformation of the input video or the original video need not becontiguous.

Moreover, in the present embodiment, while a difference vector is usedas difference information between feature amount vectors of an inputvideo and an original video, the difference information need notnecessarily be a vector as long as a difference in feature amounts foreach subregion between frame images of the input video and the originalvideo can be distinguished. In addition, while respective elements of adifference vector are denoted by “0” or “1” in the present embodiment,values in accordance with a difference in feature amounts may be usedinstead.

Returning to FIG. 1, the differing region detecting unit 24 detects adiffering region in an input video and an original video judged to haveidentity based on a difference vector stored in the differenceinformation storing unit 23 and stores detection information indicatingthe detection result in the detection information storing unit 25.

FIG. 10 is a diagram showing a configuration example of the differingregion detecting unit 24. The differing region detecting unit 24 can beconfigured so as to comprise a region mapping unit 80, a smoothing unit82, and a region detecting unit 84.

The region mapping unit 80 refers to a difference vector and maps adifference in feature amounts between frame images of an input video andan original video to a corresponding subregion for each dimension. Forexample, a dimension with a value of “1” of a difference vectorindicates that the feature amounts in a subregion corresponding to thedimension differ between the input video and the original video. Inaddition, for example, if the subregions corresponding to the dimensionare subregions 90 and 92 shown in an upper part of FIG. 11, the regionmapping unit 80 (allocating unit) adds, for example, “1” to a differencevalue of each region in the subregions 90 and 92. The region mappingunit 80 performs such a mapping process on all dimensions with adifference in feature amounts.

A difference value of each region generated by mapping performed by theregion mapping unit 80 is smoothed by the smoothing unit 82 betweenframe images and within frame images or, in other words, in temporal andspatial directions. An example of a smoothed difference value is shownin a lower part of FIG. 11.

Based on the smoothed difference value, the region detecting unit 84detects a differing region between the input video and the originalvideo, and stores detection information indicating the detection resultin the detection information storing unit 25. For example, as shown inthe lower part of FIG. 11, the region detecting unit 84 is capable ofdetecting, as a differing region, a region 94 in which the differencevalue smoothed in temporal and spatial directions projects (hereinafter,a projecting region). In this case, for example, the projecting regionmay be a region having a greater difference value than an averagedifference value of all regions. In addition, the region detecting unit84 may be configured so as to detect the projecting region as adiffering region when a size of the projecting region is greater than avalue set in advance. Alternatively, the region detecting unit 84 maydetect the projecting region as a differing region when a position of acenter of gravity of the projecting region is within a region set inadvance. Moreover, the value or the region set in advance for detectinga differing region need not necessarily be fixed and may vary inaccordance with, for example, an average difference value.

While the present embodiment is configured so that a differing regionbetween videos is detected by smoothing difference values mapped in eachframe image constituting the videos over a plurality of frames, adiffering region between frame images can be detected to a certaindegree even when using only difference values between a pair of frameimages.

In addition, in the present embodiment, while a uniform value is addedto a difference value of a region corresponding to a dimension with adifference in feature amounts regardless of the region or the dimension,the value added to difference values may vary according to the region orthe dimension. For example, when a feature amount vector extracted bythe feature amount extracting unit 20 has a characteristic for making anidentity judgment between videos while a central region of a frame imageis more heavily weighted than a surrounding region thereof, each regionor dimension may be weighted separately so that a difference in thesurrounding region is given greater consideration than a difference inthe central region when detecting a differing region.

FIG. 12 is a diagram showing an example of detection information storedin the detection information storing unit 25. As shown in FIG. 12, thedetection information includes information related to a differing regionin which a local modification has been detected. Specifically, in theexample shown in FIG. 12, the detection information includes videoidentifiers of an input video and an original video, segmentinformation, differing region information, difference information, andsimilarity information. In this case, segment information is informationindicating a video segment and, for example, a playback time or a framenumber of the segment in the video can be used. In addition, differingregion information is information indicating a position of a detecteddiffering region and, for example, information indicating regionsincluded in the differing region among the split regions shown in FIG. 3can be used. Furthermore, difference information is informationindicating a degree of difference between videos in the differingregion. Moreover, while only one numerical value is shown in FIG. 12 asdifference information related to each segment, information indicating avariation in difference in each segment can also be used. In addition,similarity information is information indicating a similarity between aninput video and an original video judged to have identity. For example,the similarity information can be outputted when the feature amountcomparing unit 70 compares feature amount vectors.

Returning to FIG. 1, the detection result output unit 26 outputsinformation indicating a differing region between an input video and anoriginal video based on the difference vector stored in the differenceinformation storing unit 23 and the detection information stored in thedetection information storing unit 25. Examples of output of informationrelated to a differing region will be described with reference to FIGS.13 to 18.

As shown in FIG. 13, the detection result output unit 26 is capable ofdisplaying a segment in which a local modification has been detected. Ascreen 110 includes a region 112 for displaying a video timeline and aregion 114 for displaying a difference between videos.

In the example shown in FIG. 13, a timeline 120 of a video is displayedin the region 112 and a segment 122 in which a differing region has beendetected is displayed on the timeline 120. Furthermore, thumbnail images124 in the segment 122 are displayed below the segment 122 in which adiffering region has been detected. In this case, for example, thedetection result output unit 26 can display the thumbnail images 124 byreferring to the video DB 32. In addition, the differing regiondetecting unit 24 may be configured so as to include, in detectioninformation, a thumbnail image of the input video in a segment in whicha differing region has been detected. In this case, the detection resultoutput unit 26 is able to use the thumbnail image included in thedetection information without having to refer to the video DB 32.

Furthermore, in the example shown in FIG. 13, a graph 130 representingdifference is displayed in the region 114. A time axis that is anabscissa of the graph 130 is consistent with a time axis of the timeline120. Therefore, as shown in FIG. 13, a segment 132 with a highdifference in the graph 130 and the segment 122 in which a differingregion has been detected and which is displayed on the timeline 120 arein a same time slot.

As shown, displaying the timeline 120 and the graph 130 representingdifference enables easy confirmation regarding in which segment adiffering region has been detected. In addition, displaying thethumbnail images 124 enables confirmation regarding in which scene of avideo the difference had occurred. Moreover, while the regions 112 and114 are displayed in the screen 110 shown in FIG. 13, only one of theregions may be displayed.

In the screen shown in FIG. 13, when the segment 122 or the segment 132indicating segments in which a differing region has been detected or thethumbnail images 124 are selected by a click or the like, the detectionresult output unit 26 outputs a screen 140 shown in FIG. 14.

The screen 140 includes regions 142 and 143 which display the originalvideo and the input video in the selected segment. The detection resultoutput unit 26 acquires segment information of the selected segment fromthe detection information, plays back the original video in the segmentfrom the video DB 32 and displays the original video in the region 142,and displays the input video in the segment in the region 143. Moreover,it is assumed that the input video is stored in a predetermined storagearea (input video storing unit) located inside or outside the differingregion detection system 10.

In addition, as shown in FIG. 14, the detection result output unit 26 iscapable of displaying frames 144 and 145 which indicate a position ofthe detected differing region on the videos displayed in the regions 142and 143. Moreover, the display of “frames” is merely an example and anydisplay method for facilitating identification of a position of adiffering region can be adopted. For example, as shown in FIG. 15, thedetection result output unit 26 may output a three-dimensional video forenabling a position of a differing region to be identifiable. FIG. 15shows videos 146-1 and 146-2 displayed in the region 142 shown in FIG.14. The video 146-1 is a left-eye video, and a region indicated by theframe 144 in FIG. 14 has been moved rightward to be displayed in aregion 147-1. In addition, the video 146-2 is a right-eye video, and aregion indicated by the frame 144 in FIG. 14 has been moved leftward tobe displayed in a region 147-2. By having the videos 146-1 and 146-2respectively displayed as a left-eye video and a right-eye video in theregion 142 shown in FIG. 14, a differing region can be stereoscopicallydisplayed. Videos displayed in the region 143 can be stereoscopicallydisplayed in a similar manner.

As described above, by displaying videos so that a position of adiffering region is identifiable, it is no longer necessary to visuallycompare all regions in the videos displayed in the regions 142 and 143when confirming a difference between the videos, and since it sufficesto only compare regions displayed so that the position of the differingregion is identifiable, work load can be reduced.

In addition, there may be cases where an input video has a plurality ofcorresponding original videos as source candidates. In such a case, thedetection result output unit 26 is capable of estimating a sourceoriginal video and displaying a segment in which a local modificationhas occurred between the estimated original video and the input video.For example, the detection result output unit 26 can display a screensuch as that shown in FIG. 16.

A screen 150 shown in FIG. 16 includes a region 152 which displaysinformation related to original videos that are source candidates and aregion 154 which displays information related to an input video. Asshown in FIG. 16, a region 156 which displays information related to theinput video is provided in the region 154. In addition, regions 158-1and 158-2 which display information related to two original videos thatare source candidates of the input video are provided in the region 152.

When there is a plurality of original videos that are source candidatesas described above, the detection result output unit 26 estimates asource original video based on detection information stored in thedetection information storing unit 25 and information stored in thefeature amount DB 30. A method of estimating a source original video canbe selected using, for example, a list box 160 or the like such as thatshown in FIG. 16. The detection result output unit 26 estimates a sourceoriginal video from the plurality of original videos according to theselected method. Examples of source estimation methods include a methodwhich gives a higher priority to videos with a longer period ofcoincidence with the input video or, in other words, a longer periodthat is judged to have identity with the input video, a method whichgives a higher priority to videos having a greater similarity with theinput video, and a method which gives a higher priority to videos withno discrepancy in a chronological order of creation dates/times withrespect to the input video. Moreover, for a similarity with the inputvideo, for example, similarity information stored in the detectioninformation storing unit 25 can be used.

In the example shown in FIG. 16, a period of coincidence with the inputvideo is selected as the source estimation method. In this case, theperiod of coincidence between the original video shown in the region158-1 and the input video is 5 minutes, and the period of coincidencebetween the original video shown in the region 158-2 and the input videois 12 minutes. Therefore, the detection result output unit 26 estimatesthe original video shown in the region 158-2 to be the source anddisplays the estimated source original video so as be identifiable by,for example, highlighting the region 158-2.

In addition, the detection result output unit 26 identifiably displays asegment in the input video in which a local modification has been madeto the estimated source original video. For example, as shown in FIG.16, the detection result output unit 26 displays the timelines of theoriginal video and the input video so that respective time axes areconsistent with each other, and displays a segment in which a localmodification has been made to the estimated source on the timeline ofthe input video. Furthermore, as shown in FIG. 16, the detection resultoutput unit 26 is capable of displaying that the modification is a“local modification” in addition to displaying the segment in which thelocal modification has been made.

In addition, when the segment in which the local modification has beenmade is selected by a click or the like in the screen 150 shown in FIG.16, the detection result output unit 26 outputs the screen 140 shown inFIG. 14 and displays an input video and an original video in theselected segment.

Furthermore, when another estimation method is selected using the listbox 160 or the like in the screen 150 shown in FIG. 16, the detectionresult output unit 26 displays a segment in which a local modificationhas occurred between the source original video estimated by the selectedestimation method and the input video. In addition, when anotheroriginal video is selected by a click or the like in the screen 150instead of a change in estimation methods, the detection result outputunit 26 similarly displays a segment in which a local modification hasoccurred by using the selected original video as a source. For example,when the region 158-1 is selected by a click or the like in in thescreen 150 shown in FIG. 16, a segment in which a local modification hasoccurred is displayed by using the original video displayed in region158-1 as a source as shown in FIG. 17.

As described above, when there is a plurality of original videos thatare source candidates of an input video, a segment in which a localmodification has occurred between an original video that is estimated tobe a source and the input video can be displayed. In addition, byselecting the displayed segment, the video in the segment can beconfirmed. Therefore, when there is a plurality of original videos thatare source candidates of an input video, the work load for confirmingcontents of the modification can be reduced.

In addition, when displaying the segment in which a local modificationhas occurred between the original video that is estimated to be a sourceand the input video, the detection result output unit 26 is capable ofmore clearly showing in which shot the modification had occurred. Forexample, the detection result output unit 26 can display a screen suchas that shown in FIG. 18.

A screen 180 shown in FIG. 18 includes a region 182 which displaysinformation related to an original video and a region 184 which displaysinformation related to an input video. As shown in FIG. 18, thumbnailimages 190 of the original video are displayed in the region 182 andthumbnail images 192 of the input video are displayed in the region 184.In this case, for example, a general method of generating thumbnailimages involves splitting a target video into shots. The shot splittingcan be performed by, for example, detecting a timing at which a featureamount vector varies significantly in a frame image included in a video.Subsequently, a thumbnail image is generated from a representative frameimage in each shot.

However, since a timing of such shot splitting often differs from atiming at which a local modification is made in a video, simplydisplaying a thumbnail image for each shot may not be sufficient forassessing contents of the modification.

In consideration thereof, with a shot in which a local modification hasbeen detected among shots split by a general method, the detectionresult output unit 26 can further perform shot splitting in accordancewith a presence/absence of a local modification to generate a thumbnailimage.

For example, let us assume that in the screen shown in FIG. 18, threeshots 194, 196, and 198 are produced as a result of shot splittingperformed by a general method. As shown in FIG. 18, assuming that alocal modification has been detected in each shot, by further performingshot splitting in accordance with a presence/absence of a localmodification in each shot, the detection result output unit 26 iscapable of generating a shot that coincides with a timing at which thelocal modification has been made. In addition, for example, thedetection result output unit 26 is capable of displaying a thumbnailimage 200 of a shot at the timing at which the local modification hasbeen made. Furthermore, the detection result output unit 26 can displayinformation 202 that is capable of identifying that a “localmodification” has occurred in the segment together with the thumbnailimage 200 of the shot at the timing at which the local modification hasbeen made. The same applies for the other shots 196 and 198.

Moreover, a general shot splitting process may be performed before inputto the differing region detection system 10. Alternatively, a generalshot splitting process may be performed by a shot splitting unitprovided inside the differing region detection system 10.

As described above, by further performing shot splitting according tothe presence/absence of a local modification on the inside of a shot inwhich a local modification has occurred, an assessment regarding inwhich shot the modification has occurred can be made more readily.Accordingly, the work load when confirming modification contents can bereduced.

FIG. 19 is a flow charting an example of a differing region detectingprocess according to the differing region detection system 10. First,the feature amount extracting unit 20 extracts a feature amount vectorfor each frame image in an input video and stores the feature amountvectors in the feature amount storing unit 21 (S1901).

The feature amount comparing unit 70 compares the feature amount vectorof the input video stored in the feature amount storing unit 21 with afeature amount vector of an original video stored in the feature amountDB 30 (S1902). The frame selecting unit 72 selects a frame image havingidentity based on a result of the comparison by the feature amountcomparing unit 70 (S1903). In addition, the difference informationoutput unit 74 stores a difference vector for the selected frame imagein the difference information storing unit 23 (S1904).

The region mapping unit 80 maps the difference value to a regioncorresponding to a dimension in which a difference in feature amountshas occurred based on the difference vector stored in the differenceinformation storing unit 23 (S1905). The smoothing unit 82 smooths themapped difference value in temporal and spatial directions (S1906). Inaddition, based on the smoothed difference value, the region detectingunit 84 detects a differing region between the input video and theoriginal video, and stores detection information in the detectioninformation storing unit 25 (S1907).

Finally, based on the detection information stored in the detectioninformation storing unit 25, the detection result output unit 26 outputsinformation indicating a differing region between the input video andthe original video having identity (S1908).

This concludes the description of the differing region detection system10 according to the present embodiment. With the differing regiondetection system 10, by mapping a difference in feature amounts for eachdimension of a feature amount vector to a subregion corresponding toeach dimension instead of simply comparing distances between featureamount vectors, a differing region between videos with identity can bedetected.

In addition, with the differing region detection system 10, a segmentwith identity between compared videos can be specified and a differingregion in the specified segment can be detected.

Furthermore, with the differing region detection system 10, since adifference value mapped to a subregion corresponding to a dimension withvalues that differ in a difference vector is smoothed in temporal andspatial directions, a differing region can be detected with highprecision.

Moreover, with the differing region detection system 10, a differingregion can be detected while taking a weight set to each dimension oreach region in a difference vector into consideration. For example, whena feature amount vector used to judge identity more greatly reflects afeature amount in a central portion in an image region as compared to asurrounding portion, the weight of a region in the surrounding portionmay be increased when detecting a differing region. For example, sincetelops are often added to a lower portion of an image region, increasingweight of a region in the lower portion is effective when detecting adiffering region between videos with a difference in telops. Inaddition, for example, a difference in an outermost circumferentialportion of an image region is likely to increase even when there isidentity but no local difference between videos. Therefore, weight ofthe outermost circumferential portion of the image region may bereduced.

In addition, with the differing region detection system 10, a positionof a detected differing region can be displayed so as to beidentifiable. Accordingly, a user can readily confirm the position of adiffering region between videos with identity.

Furthermore, with the differing region detection system 10, a segment inwhich a differing region has occurred in a video can be displayed so asto be identifiable. Therefore, when confirming contents that differbetween videos, since a user need only confirm videos of the segmentinstead of entire videos, work load can be reduced.

It should be noted that the present embodiment is for facilitatingunderstanding of the present invention and is not for limiting theinterpretation of the present invention. Various modifications andchanges may be made to the present invention without departing from thespirit and scope thereof, and equivalents are to be included in thepresent invention.

The present application claims priority on the basis of Japanese PatentApplication No. 2011-027429 filed on Feb. 10, 2011, the entire contentsof which are incorporated herein by reference.

While the present invention has been described with reference to anembodiment, the present invention is not intended to limit theembodiment described above. Various modifications to configurations anddetails of the present invention will occur to those skilled in the artwithout departing from the scope of the present invention.

A part of or all of the present embodiment may also be described as, butnot limited to, the appendices provided below.

(Appendix 1) A differing region detection system, comprising: adifference information generating unit configured to generateinter-image difference information indicating a difference in featureamounts for each subregion between first and second images based on afirst feature amount vector that is a set of feature amountsrespectively corresponding to a plurality of subregions in the firstimage and a second feature amount vector that is a set of featureamounts respectively corresponding to a plurality of subregions in thesecond image; and a differing region detecting unit configured to detecta differing region that is an image region that differs between thefirst and second images, based on differences in the respectivesubregions indicated by the inter-image difference information, andoutput detection information indicating a result of the detection.(Appendix 2) The differing region detection system according to Appendix1, wherein the subregions include at least one split region, and thediffering region detecting unit is configured to detect the differingregion with the split region as a unit by allocating a difference valuein accordance with the difference to each split region in each subregionbased on the inter-image difference information.(Appendix 3) The differing region detection system according to Appendix1, wherein the first image is a first frame image constituting a firstvideo, the second image is a second frame image constituting a secondvideo and corresponding to the first frame image, the differenceinformation generating unit is configured to generate the inter-imagedifference information for a plurality of pairs of the first and secondimages, and the differing region detecting unit is configured to detectthe differing region in the first and second videos based on theinter-image difference information for the plurality of pairs of thefirst and second images.(Appendix 4) The differing region detection system according to Appendix3, wherein the subregions include at least one split region, and thediffering region detecting unit is configured to: allocate a differencevalue in accordance with the difference to each split region in eachsubregion based on the inter-image difference information; and detectthe differing region with the split region as a unit by smoothing thedifference value for the plurality of pairs of the first and secondimages between frame images.(Appendix 5) The differing region detection system according to Appendix3 or 4, wherein the differing region detecting unit is configured todetect the differing region by smoothing the difference value for theplurality of pairs of the first and second images between frame images.(Appendix 6) The differing region detection system according to any oneof Appendices 3 to 5, wherein the difference information generating unitis configured to: select a plurality of pairs of the first and secondimages in which a difference in feature amount vectors is smaller than apredetermined criterion based on a plurality of the first feature amountvectors and a plurality of the second feature amount vectors; and outputthe inter-image difference information for the selected plurality ofpairs.(Appendix 7) The differing region detection system according to any oneof Appendices 3 to 6, wherein the differing region detecting unit isconfigured to detect the differing region based on a weight set for eachof the subregions and the difference value for the plurality of pairs ofthe first and second images.(Appendix 8) The differing region detection system according to any oneof Appendices 3 to 7, wherein the difference information generating unitis configured to select a plurality of pairs of the first and secondimages among the first video and one second video among the plurality ofsecond videos, based on the plurality of first feature amount vectors ofthe first video and the plurality of second feature amount vectors ofeach of the plurality of second videos.(Appendix 9) The differing region detection system according to Appendix8, wherein the difference information generating unit is configured toselect a plurality of pairs of the first and second images among onefirst video among the plurality of first videos and one second videoamong the plurality of second videos, based on the plurality of firstfeature amount vectors of each of the plurality of first videos and theplurality of second feature amount vectors of each of the plurality ofsecond videos.(Appendix 10) The differing region detection system according to any oneof Appendices 1 to 9, further comprising a detection result output unitconfigured to identifiably display a position of the differing regionbetween the first and second images based on the detection information.(Appendix 11) The differing region detection system according to any oneof Appendices 3 to 9, further comprising a detection result output unitconfigured to identifiably display a position of the differing regionbetween the first and second videos based on the detection information.(Appendix 12) The differing region detection system according toAppendix 11, wherein the differing region detecting unit is configuredto include information indicating a segment in which the differingregion has been detected between the first and second videos in thedetection information and output the information, and the detectionresult output unit is configured to identifiably display the segment inwhich the differing region has been detected based on the detectioninformation.(Appendix 13) The differing region detection system according toAppendix 11 or 12, wherein the differing region detecting unit isconfigured to include information indicating a degree of difference inthe differing region in the detection information and output theinformation, and the detection result output unit is configured toidentifiably display the degree of difference in the differing regionbased on the detection information.(Appendix 14) The differing region detection system according toAppendix 12, wherein in response to a user input for selecting a segmentin which the differing region has been detected, the detection resultoutput unit is configured to display the first and second videos in thatsegment.(Appendix 15) A differing region detection method, comprising the stepsof: generating inter-image difference information indicating adifference in feature amounts for each subregion between first andsecond images based on a first feature amount vector that is a set offeature amounts respectively corresponding to a plurality of subregionsin the first image and a second feature amount vector that is a set offeature amounts respectively corresponding to a plurality of subregionsin the second image; and detecting a differing region that is an imageregion that differs between the first and second images, based ondifferences in the respective subregions indicated by the inter-imagedifference information and outputting detection information thatindicates a result of the detection.

-   10 differing region detection system-   20 feature amount extracting unit-   21 feature amount storing unit-   22 difference information generating unit-   23 difference information storing unit-   24 differing region detecting unit-   25 detection information storing unit-   26 detection result output unit-   30 feature amount database-   32 video database-   70 feature amount comparing unit-   72 frame selecting unit-   74 difference information output unit-   80 region mapping unit-   82 smoothing unit-   84 region detecting unit

1. A differing region detection system, comprising: a differenceinformation generating unit configured to generate inter-imagedifference information indicating a difference in feature amounts foreach subregion between first and second images based on a first featureamount vector that is a set of feature amounts respectivelycorresponding to a plurality of subregions in the first image and asecond feature amount vector that is a set of feature amountsrespectively corresponding to a plurality of subregions in the secondimage; and a differing region detecting unit configured to detect adiffering region that is an image region that differs between the firstand second images, based on differences in the respective subregionsindicated by the inter-image difference information, and outputdetection information indicating a result of the detection.
 2. Thediffering region detection system according to claim 1, wherein thesubregions include at least one split region, and the differing regiondetecting unit is configured to detect the differing region with thesplit region as a unit by allocating a difference value in accordancewith the difference to each split region in each subregion based on theinter-image difference information.
 3. The differing region detectionsystem according to claim 1, wherein the first image is a first frameimage constituting a first video, the second image is a second frameimage constituting a second video and corresponding to the first frameimage, the difference information generating unit is configured togenerate the inter-image difference information for a plurality of pairsof the first and second images, and the differing region detecting unitis configured to detect the differing region in the first and secondvideos based on the inter-image difference information for the pluralityof pairs of the first and second images.
 4. The differing regiondetection system according to claim 3, wherein the subregions include atleast one split region, and the differing region detecting unit isconfigured to: allocate a difference value in accordance with thedifference to each split region in each subregion based on theinter-image difference information; and detect the differing region withthe split region as a unit by smoothing the difference value for theplurality of pairs of the first and second images between frame images.5. The differing region detection system according to claim 3, whereinthe differing region detecting unit is configured to detect thediffering region by smoothing the difference value for the plurality ofpairs of the first and second images between frame images.
 6. Thediffering region detection system according to claim 3, wherein thedifference information generating unit is configured to: select aplurality of pairs of the first and second images in which a differencein feature amount vectors is smaller than a predetermined criterionbased on a plurality of the first feature amount vectors and a pluralityof the second feature amount vectors; and output the inter-imagedifference information for the selected plurality of pairs.
 7. Thediffering region detection system according to claim 3, wherein thediffering region detecting unit is configured to detect the differingregion based on a weight set for each of the subregions and thedifference value for the plurality of pairs of the first and secondimages.
 8. The differing region detection system according to claim 3,wherein the difference information generating unit is configured toselect a plurality of pairs of the first and second images among thefirst video and one second video among the plurality of second videos,based on the plurality of first feature amount vectors of the firstvideo and the plurality of second feature amount vectors of each of theplurality of second videos.
 9. The differing region detection systemaccording to claim 8, wherein the difference information generating unitis configured to select a plurality of pairs of the first and secondimages among one first video among the plurality of first videos and onesecond video among the plurality of second videos, based on theplurality of first feature amount vectors of each of the plurality offirst videos and the plurality of second feature amount vectors of eachof the plurality of second videos.
 10. The differing region detectionsystem according to claim 1, further comprising a detection resultoutput unit configured to identifiably display a position of thediffering region between the first and second images based on thedetection information.
 11. The differing region detection systemaccording to claim 1, further comprising a detection result output unitconfigured to identifiably display a position of the differing regionbetween the first and second videos based on the detection information.12. The differing region detection system according to claim 11, whereinthe differing region detecting unit is configured to include informationindicating a segment in which the differing region has been detectedbetween the first and second videos in the detection information andoutput the information, and the detection result output unit isconfigured to identifiably display the segment in which the differingregion has been detected based on the detection information.
 13. Thediffering region detection system according to claim 11, wherein thediffering region detecting unit is configured to include informationindicating a degree of difference in the differing region in thedetection information and output the information, and the detectionresult output unit is configured to identifiably display the degree ofdifference in the differing region based on the detection information.14. The differing region detection system according to claim 12, whereinin response to a user input for selecting a segment in which thediffering region has been detected, the detection result output unit isconfigured to display the first and second videos in that segment.
 15. Adiffering region detection method, comprising the steps of: generatinginter-image difference information indicating a difference in featureamounts for each subregion between first and second images based on afirst feature amount vector that is a set of feature amountsrespectively corresponding to a plurality of subregions in the firstimage and a second feature amount vector that is a set of featureamounts respectively corresponding to a plurality of subregions in thesecond image; and detecting a differing region that is an image regionthat differs between the first and second images, based on differencesin the respective subregions indicated by the inter-image differenceinformation and outputting detection information that indicates a resultof the detection.